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achieve greater results for the American people. This project specifically has three key objectives: 

• Eliminating or redesigning misguided spending programs and tax expenditures, focused on priority areas 
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Introduction and summary 


This report is the culmination of a yearlong effort to study the efficiency of the 
nation’s public education system and includes the first-ever attempt to evaluate the 
productivity of almost every major school district in the country. In the business 
world, the notion of productivity describes the benefit received in exchange for 
effort or money expended. Our project measures the academic achievement a school 
district produces relative to its educational spending, while controlling for factors 
outside a district’s control, such as cost of living and students in poverty. 

Our nation’s school system has for too long failed to ensure that education funding 
consistently promotes strong student achievement. After adjusting for inflation, educa- 
tion spending per student has nearly tripled over the past four decades. 1 But while some 
states and districts have spent their additional dollars wisely — and thus shown significant 
increases in student outcomes — overall student achievement has largely remained flat. 2 
And besides Luxembourg, the United States spends more per student than any of the 
65 countries that participated in a recent international reading assessment, and while 
Estonia and Poland scored at the same level as the United States on the exam, the United 
States spent roughly $60,000 more to educate each student to age 15 than either nation. 3 

Our aims for this project, then, are threefold. First, we hope to kick-start a national 
conversation about educational productivity. Second, we want to identify districts that 
generate higher-than-average achievement per dollar spent, demonstrate how pro- 
ductivity varies widely within states, and encourage efforts to study highly productive 
districts. Third — and most important — we want to encourage states and districts to 
embrace approaches that make it easier to create and sustain educational efficiencies. 

This report comes at a pivotal time for schools and districts. Sagging revenues have 
forced more than 30 states to cut education spending since the recession began. 4 The 
fiscal situation is likely to get worse before it gets better because the full impact of the 
housing market collapse has yet to hit many state and local budgets. 5 At a time when 
states are projecting more than $100 billion in budget shortfalls, educators need to be 
able to show that education dollars produce significant outcomes or taxpayers might 
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begin to see schools as a weak investment . 6 If schools don’t deliver maximum 
results for the dollar, public trust in education could erode and taxpayers may 
fund schools less generously. 

While some forward-thinking education leaders have taken steps to promote bet- 
ter educational efficiency most states and districts have not done nearly enough 
to measure or produce the productivity gains our education system so desperately 
needs. Some fear that a focus on efficiency might inspire policymakers to reduce 
already limited education budgets and further increase the inequitable distribu- 
tion of school dollars. To be sure, our nation’s system of financing schools is unfair. 
Low-income and minority students are far more likely to attend schools that don’t 
receive their fair share of federal, state, and local dollars. But while the issue of 
fairness must be central to any conversation about education finance, efficiency 
should not be sacrificed on the altar of equity. Our nation must aspire to have a 
school system that’s both fair and productive. 

Our emphasis on productivity does not mean we endorse unfettered market-based 
reforms, such as vouchers allowing parents to direct public funds to private schools. 
Nor do we argue that policymakers should spend less on education. Indeed, we 
believe neither of these approaches can solve the nation’s pressing education chal- 
lenges. Transforming our schools will demand both real resources and real reform. 
As Education Secretary Arne Duncan recently said: “It’s time to stop treating the 
problem of educational productivity as a grinding, eat-your-broccoli exercise. It’s 
time to start treating it as an opportunity for innovation and accelerating progress .” 7 


Productivity 101: Embracing transformational change 


When successful businesses want to improve performance and boost 
efficiencies, they focus on creating the conditions for organizational 
change. They use data to identify problem areas, create short and 
long-term goals, and engage their employees to sustain transforma- 
tions and nurture further innovation. Such approaches have long 
worked for the private sector, and there's clear evidence that the tech- 
niques can help drive better performance in large, public organiza- 
tions as well. 8 


That will entail doing away with obsolete traditions and ineffective 
programs, to be sure. But it will also require schools and districts to 
embrace transformational ways of delivering a cost-effective educa- 
tion that reduces spending while boosting performance. The goal 
must be nothing short of a breakthrough in performance that guar- 
antees that every dollar produces high achievement for all students. 


their efficacy. 9 To increase productivity, school leaders will need to 
fundamentally reinvent the way that they do business and create 
an outcomes-based school culture that sets high goals — and gives 
employees the strategies to achieve them. 


ones and dash from one educational fad to the next without tracking 


But schools and districts have long been effective at deflecting or 
watering down meaningful change in order to protect entrenched 
bureaucracies and interests. And even reform-minded school ad- 
ministrators often confuse merely novel techniques with successful 
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The Wisconsin 


Accompanying this report is an interactive website that allows anyone to compare 
the relative productivity of thousands of school districts and find out more about 
their spending and achievement. Because we cannot control for everything outside 
a district's control when calculating its productivity evaluation, the site makes it easy 
to compare similar districts based on their demographics and enrollment. It also al- 
lows users to see how districts fare under different approaches to measuring produc- 
tivity. The site was created in partnership with the nonprofit advocacy organization 
OMB Watch and the geo-information services company Esri. 


Return on Educational Investment: A district-by-district evaluation of educational productivity 

To spark a national dialogue about educational productivity, we've attempted to evaluate the return on investment (ROI) of almost every major school district in the country. 

By productivity, we mean how much learning a district produces for every dollar spent, after controlling for factors such as cost of living and students in poverty. Use the 
map to see how each district performs and click on the tabs to toggle between our three different approaches. We call them Basic ROI, Adjusted ROI, and Predicted 
Index. Click here for our methodology and here for additional instructions on how best to use the site. 



Virginia 

Identify high-achieving, low-spending school distncts 

100 

BEST 



School District 

Basic ROI 

State Achiewsment Index 

Adj. Per Pupil Spending 

Percent Low Income 

Enrollment 

Urbanicity 


Amelia County Public Schools 

MH 

85 

$6,732 

40 

1,880 

Rural: Distant 

A 

Campbell County Public Schools 

Hi 

88 

$7,493 

29 

8,813 

Rural: Fringe 


Floyd County Public Schools 


90 

$7,366 

37 

2,059 

Rural: Distant 


Am herst County Public Schools 


85 

$7,731 

42 

4,764 

Rural: Fringe 


Grayson County Public Schools 


81 

$8,430 

54 

2,109 

Rural: Remote 


Hanover County Public Schools 


94 

$7,239 

8 

19,100 

Rural: Fringe 


Accomack County Public Schools 


89 

$7,718 

61 

5,206 

Rural: Remote 


Bland County Public Schools 


88 

$8,114 

36 

906 

Rural: Remote 


Brunswick County Public Schools 

■■ 

78 

$7,988 

76 

2,256 

Rural: Fringe 


Charlotte County Public Schools 


92 

$7,798 

50 

2,195 

Rural: Remote 

V 


* Per pupil spending has been adjusted for differences in cos! of living ard students in special orogarms. 

Sources: For spending and demographic information, we relied on 2008 data comcvec by tne U.S. Department of Education's National Center for Educatcn Statistics. For overall school district achievement results, we used 2008 data 
compiled by the New Arrenca Foundation s Federal Education Budget Project. Some states and districts were missing data for certain demographic indicators, necessitating the use of proxies. For more information on our sources, use 
of proxies, and our approach to calculating educational productivity, please consult our detailed methodology. 


school systems of 
Oshkosh and Eau 
Claire are about the 
same size and serve 
similar student 
populations. They 
also get largely 
similar results on 
state exams-but 
Eau Claire spends 
an extra $8 million 
to run its school 
system. 
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Summary of findings 


• Many school districts could boost student achievement without increasing 
spending if they used their money more productively. An Arizona school 
district, for example, could see as much as a 36 percent boost in achievement if 
it increased its efficiency from the lowest level to the highest, all else being equal. 

• Low productivity costs the nation's school system as much as $1 75 billion a 
year. This figure is an estimate; our study does not capture everything that goes 
into creating an efficient district. But the approximate loss in capacity equals 
about 1 percent of the nations gross domestic product. 10 

• Without controls on how additional school dollars are spent, more education 
spending will not automatically improve student outcomes. In more than 
half of the states included in our study, there was no clear relationship between 
spending and achievement after adjusting for other variables, such as cost of liv- 
ing and students in poverty. These findings are consistent with existing research: 
How a school system spends its dollars can be just as important as how much it 
spends, at least above some threshold level. 

• Efficiency varies widely within states. Some districts spent thousands more 
per student to obtain the same broad level of academic achievement. After 
adjusting for factors outside of a district’s control, the range of spending 
among the districts scoring in the top third of achievement in California was 
nearly $8,000 per student. 

• More than a million students are enrolled in highly inefficient districts. Over 
400 school districts around the country were rated highly inefficient on all three 
of our productivity metrics. These districts serve about 3 percent of the almost 
43 million students covered by our study. 

• High-spending school systems are often inefficient. Our analysis showed 
that after accounting for factors outside of a district’s control, many high- 
spending districts posted middling productivity results. For example, only 17 
percent of Florida’s districts in the top third in spending were also in the top 
third in achievement. 
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• Students from disadvantaged backgrounds are more likely to be enrolled 
in highly inefficient districts. Students who participated in subsidized lunch 
programs were 12 percentage points more likely to be enrolled in the nation’s 
least-productive districts, even after making allowances for the higher cost of 
educating lower-income students. 11 

• Highly productive districts are focused on improving student outcomes. We 

surveyed a sample of highly productive districts to learn more about their 
principles and practices. The districts that performed well on our metrics shared 
a number of values and practices, including strong community support and a 
willingness to make tough choices. 

• States and districts fail to evaluate the productivity of schools and districts. 

While the nation spends billions of dollars on education, only two states, Florida 
and Texas, currently provide annual school-level productivity evaluations, which 
report to the public how well funds are being spent at the local level. 

• The quality of state and local education data is often poor. In many instances, 
key information on school spending and outcomes is not available or insuffi- 
ciently rigorous, and this severely impedes the study of educational productiv- 
ity. For instance, we did not have good enough data to control for certain cost 
factors, such as transportation. So a rural district with high busing costs might 
suffer in some of our metrics compared with a more densely populated district. 

• The nation's least-productive districts spend more on administration. The 

most inefficient districts in the country devote an extra 3 percentage points of 
their budgets on average to administration, operations, and other noninstruc- 
tional expenditures. 

• Some urban districts are far more productive than others. While our main 
results are limited to within-state comparisons, we were able to conduct a spe- 
cial cross-state analysis of urban districts that recently participated in a national 
achievement test. After adjusting for certain factors outside a district’s control, 
we found that some big-city school systems spend millions of dollars more than 
others — but get far lower results on math and reading tests. 
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Summary of recommendations 


Policymakers should promote educational efficiency 

We hope this report launches a broad dialogue about educational productivity. 
Education policymakers should encourage further research in this area, as well as 
convene a national panel to recommend how state and federal governments can 
better support policies and programs that promote efficiency. 


States and districts must reform school management systems 

Education policymakers should create performance-focused management systems 
that are flexible on inputs and strict on outcomes. State and federal governments 
should also provide educators with the tools, technology, and training required to 
succeed with limited school dollars. 


Education leaders should encourage smarter, fairer approaches to school funding 

Education policymakers should develop funding policies that direct money to stu- 
dents based on their needs, so that all schools and districts have an equal opportu- 
nity to succeed. Federal policymakers should also continue to support competitive 
funding programs that create opportunities for reform and innovation. 


States and districts should report far more data on school performance 

States and districts should develop data systems that report reliable, high-quality 
information on educational outcomes, operations, and finance. 
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Productivity ratings used in this study 


Our work on educational productivity builds on the 2007 "Leaders 
and Laggards" report released by the Center for American Progress 
in partnership with the U.S. Chamber of Commerce and Frederick 
Hess of the American Enterprise Institute. In that study, we evaluated 
state-level return on investment (ROI), comparing state achievement 
outcomes with education spending, after controlling for differences 
in special education and low-income students, and living costs. 

For this study, we developed three district-level productivity mea- 
sures. We relied on spending data from the 2008 school year, the 
most recent available. For achievement, we relied on the results of 
2008 state reading and math assessments in fourth grade, eighth 
grade, and high school. We believe this is the first national effort to 
gauge the efficiency of over 9,000 districts in more than 45 states 
against a set of evaluative rubrics. All three of our metrics use a 
green-to-red color-coding system, and the first two approaches use 
the matrix shown below to evaluate districts. The same color legend 
is used on the interactive companion website at www.american- 
progress.org/ROI. 

ROI Evaluation Matrix 

Lowest Medium Highest 

achievement achievement achievement 

Lowest cost 
Medium cost 
Highest cost 


Basic Return on Investment index rating 

This measure rates school districts on how much academic achieve- 
ment they get for each dollar spent, relative to other districts in their 
state. To avoid penalizing districts where education costs are higher, 
we adjusted for a variety of factors including cost-of-living differences 
as well as higher concentrations of low-income, non-English-speak- 
ing, and special education students. 


Adjusted Return on Investment index rating 

This measure uses the same approach as the Basic ROI but applies a 
different statistical method, called a regression analysis, to account 
for the higher costs associated with serving larger concentrations of 
low-income, non-English-speaking, and special education students. 
The adjustments, or weights, used in the Basic ROI are not always 
sensitive enough to account for spending differences within states. 

Predicted Efficiency index rating 

The Predicted Efficiency rating measures whether a district's achieve- 
ment is higher or lower than would be predicted after accounting for 
its per-pupil spending and concentrations of low-income, non-English- 
speaking, and special education students. Linder this approach, a 
low-achieving district could get high marks if it performed better than 
predicted. Lowering academic expectations for students from disad- 
vantaged backgrounds is not a policy position supported by the Center. 

Our measures are far from perfect, and individual district evaluations 
should be interpreted with caution. The connection between spend- 
ing and achievement is complex, and our methods cannot capture 
everything that goes into creating an efficient school system. Nor can 
we control for everything that's outside of a district's control, and our 
adjustments for factors like poverty and students in special education 
are estimations and don't account for variations in severity and type 
within those demographic groups. Moreover, most of the variation in 
student achievement is within schools, and so district-level produc- 
tivity results most likely mask significant variations in productivity 
within districts. Finally, we're aware that some of the data reported by 
states and districts have reliability issues, with agencies sometimes 
using inconsistent definitions and weak data collection practices. 

Despite these important caveats, we believe our district-level ratings 
use the best available methods and reveal important results. Our 
work has been aided by a panel of experts, who reviewed our ap- 
proach and provided helpful feedback. However, we take full respon- 
sibility for the methodology and resulting evaluations. 



Introduction and summary | www.americanprogress.org 7 


America’s ingenuity and dynamism have helped make it one of the most prosper- 
ous countries in the world; and these traits will be key to boosting the productiv- 
ity of our school system. Efficiency reforms will not come fast or easy, but they 
must come soon because our nation can no longer afford schools that fail to 
make the most of their limited resources. We hope that our interactive website — 
together with the findings and recommendations in the following pages — will 
prompt a new way of thinking about educational success, one that does far more 
to guarantee that all schools and districts have the necessary data, tools, and 
incentives to be efficient with their school dollars. 
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Background 


In education, spending does not always equal success. Countless studies have 
shown that how a school system spends its dollars can be just as important as how 
much it spends. But our country’s education system lacks the proper incentives, 
support, and accountability structures to ensure that resources deliver the most effi- 
cient results. This section explains how we arrived at this point and what we must 
do to reform. We also detail our methods of evaluating educational productivity. 


A brief history of educational productivity reform 

Attempts to improve the nation’s educational productivity date back at least to the 
19 th century, when towns and cities were the major funders of public schools. As 
states increasingly took responsibility for local education systems, they often tried 
to organize schools around business ideas in vogue at the time, such as applying 
“scientific management” theories to increase labor productivity. 12 

Education leaders have since tried a variety of reforms to inject greater efficiencies 
into the nation’s education system, from creating school districts to mandating 
instructional spending as a budget priority. But for the most part, these initiatives 
have focused on improving administration and operations, and so they typi- 
cally require a one-size-fits-all solution that ultimately limits a school’s ability to 
creatively deploy resources. And the data suggests that the strategies have had little 
effect on improving overall productivity. Academic achievement per school dollar 
dropped by as much as 65 percent from 1971 to 1999. 13 Despite our large annual 
education expenditures, the United States gets far less for its dollar than do other 
industrialized countries. 

In many ways, the issue boils down to the school system’s long-standing focus 
on inputs instead of outcomes, and states for many years were reluctant to even 
outline what students should learn before graduating from high school. There 
has been significant recent progress in this area, however. In the last two decades, 
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"We need to see the 
possibilities. When 
there is an outlier 
in the district — an 
upward outlier — 
that means the 
bar is raised and 
others can do it," 
said Susan Parks, 
superintendent of 
San Gabriel Unified 
School District near 
Los Angeles, which 
received high marks 
on our productivity 
metrics. 


every state has adopted academic standards that describe what students should 
know and be able to do. States have also created assessment programs that hold 
schools and districts accountable for their performance against those standards. 

The development of academic standards has important implications for advocates 
of educational productivity. Standards make it far easier to evaluate productivity 
because all school systems within a state now work toward a common educational 
goal. The measures also allow educational management systems to better hold 
schools and districts accountable for their results. 

Some education experts remain hesitant about judging schools by how much 
achievement they produce relative to their spending. They are concerned that it 
is difficult to adjust for all the factors outside a district’s control. For example, the 
way some states currently count students in special education programs doesn’t 
reliably separate students with severe disabilities from those with less debilitat- 
ing conditions. That makes it hard to know how much additional effort or money 
educating a student with special needs might entail. 

Another concern is that a focus on productivity might inspire policymakers to 
reduce the already limited funds that schools receive. This concern is particularly 
acute with regard to schools with large enrollments of low-income and minority 
students, many of which already receive far less than their fair share of federal, state, 
and local dollars. Indeed, many states still have regressive funding systems in which 
high-poverty districts get less money than wealthier ones. In New Hampshire, for 
instance, a district with a 30 percent poverty rate receives about two-thirds the 
money per student than a district with no student poverty. 14 Other federal and local 
fiscal policies exacerbate the problem, and on average, the highest poverty districts 
receive $825 per student less in revenues than the most affluent districts. 15 

The Center for American Progress has long advocated for greater fiscal equity 
across schools and districts, and we’ve gone to great lengths in this study to level 
the playing field and adjust for differences among districts with different popula- 
tions of students. But equity and efficiency are not mutually exclusive, and our 
nation’s school system should aspire to be both fair and productive. Moreover, 
there are unambiguous examples of schools with high concentrations of low- 
income students that have significant resources but fail to make reasonable 
academic progress, and if there are not productivity reforms made to the nation’s 
education system, it’s clear that low-income and minority students will continue 
to be harmed by the ways in which schools and districts are funded. 
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What we don't know: A blind eye to educational productivity 


In many ways, the nations educational productivity problem is an educational 
management problem. Our school system lacks a performance-focused set of 
practices and policies that ensure that dollars are spent efficiently. The problem 
manifests itself in ways both big and small. There are the examples of straightfor- 
ward waste, of course: overpaying for food services by not considering outside 
providers; students being taught the same material twice because of poorly orga- 
nized educational programs; central offices becoming bloated and broken due to 
operational neglect. 

The bigger issue is that state and local school operations don’t provide educators 
with the tools, skills, and incentives to connect spending to outcomes and reorga- 
nize inefficient programs. 16 For instance, many states use seat-time requirements 
to determine whether a student is ready to graduate from high school. While such 
a requirement may have made sense at one time, it divorces inputs from outcomes 
and prevents educators from trying more productive ways to make all students 
college- and career-ready. 

Local leaders also have little autonomy to spend dollars in ways that they believe 
will bring the most results. In most areas, the district oversees the school’s budget 
and has the final say on spending decisions. Teacher salaries are also often set at 
the state level, preventing educators from linking teacher salary to effectiveness. 
And there’s striking uniformity about how districts budget their dollars; the vast 
majority allocate about 60 percent to student instruction. 17 

The nation’s haphazard approach to school finance exacerbates the problem, with 
few states and districts structuring their spending programs to make them as effi- 
cient as possible. States often have dozens of different funding programs, and each 
of these spending initiatives usually has its own set of specific rules and regula- 
tions. Take California, which earmarked more than $41 million in 2009 to hire 
additional gym teachers to combat childhood obesity. This despite there being 
no shortage of gym instructors in the state, or any evidence that increasing the 
number of gym teachers reduces obesity. Worse, the grants are not even targeted 
at schools with large percentages of overweight students. 18 

Many states and districts don’t even bother to collect data on program effectiveness. 
A recent study found that Philadelphia spent $ 162 million on professional devel- 
opment in 2008, or about $6,000 per teacher. But more than six different offices 
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oversaw the investments; and there was no systematic effort to measure the need for 
the initiatives. 19 Such a lack of data makes it difficult to pinpoint specific inefficien- 
cies. It also underscores the degree to which the nation’s educational system lacks 
the accountability structures needed to ensure that resources deliver results. 


What we do know: The relationship between money and results 

The relationship between school funding and academic achievement is a subject 
of much debate within the education policy community. Some experts maintain 
it’s possible to cut education funding without lowering achievement, while others 
argue that only an influx of more money can bring the achievement boost our 
schools so desperately need. 

The research does make a few things clear. First, the literature strongly calls into 
question the notion that simply investing more money in schools will result in 
better outcomes. At least above some threshold level, the research suggests that if 
policymakers allocate additional education dollars without any clear controls on 
how the money is spent, the funds do not appear to have a significant impact on 
achievement. 20 But the literature also makes plain that school spending can make 
a difference in achievement; a large body of research shows that certain inputs 
such as teacher quality can significantly impact student outcomes. One series of 
studies showed that students who have three or four highly effective teachers in a 
row will succeed academically, while those who have a sequence of weak teachers 
fall further behind. 21 Another study found that raising teacher wages by 1 0 percent 
could reduce high school dropout rates by more than 3 percent. 22 

The bottom line: Additional dollars make a difference only if the funds are well 
spent. As Stanford University economist Eric Hanushek said in a recent interview: 
“In some places, you put money in and you get results. In other places, you put 
money in and you don’t get any results. It’s not that money can’t matter, and it’s 
not that it doesn’t matter in some circumstances. It’s just that if you do what the 
courts talk about, what legislatures often talk about, which is a helicopter drop of 
large amounts of money into districts with no expectations for how well it will be 
spent, you don’t see much coming out the other end.” 23 

The research also offers clues to why the nation’s school productivity remains so 
low. The problem is that U.S. academic achievement has remained largely flat even 
as educational expenses have been skyrocketing, with inflation-adjusted per student 
spending increasing 60 percent since 1985. 24 Part of the issue is that schools employ 
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far more people than ever before, and the teacher force has increased at more 
than double the rate of enrollment. 25 The cost per teacher has also been escalating 
because of the rising expense of pensions and health care, and the costs of total ben- 
efits rose from 25 percent of teacher salaries in 1999 to almost a third in 2006. 26 

To be sure, school systems have by no means wasted the additional dollars that they 
have spent over the past four decades. Some of the cost increases, like health care 
expenditures, were beyond the control of schools and districts, and a number of 
states, such as Massachusetts, have shown that strategic spending can make a large 
and significant difference in student achievement. Additional funds also appear to 
have helped narrow the achievement gap between minority and white students, 
and from 1973 to 2008 the math scale scores of African-American 4th graders 
jumped by more than 30 points, or roughly three grade levels. 27 Still, despite mas- 
sive increases in expenditures, overall student outcomes have remained largely 
stagnant, and achievement gaps remain wide in many areas. American taxpayers, 
in other words, have seen only a small return on the dollars they’ve invested in the 
nations school system over the past 40 years. This can and must change. 


The bottom line: 
Additional dollars 
make a difference 
only if the funds are 
well spent. 


The case for focusing on educational productivity now 

The economic downturn has dramatically changed the fiscal climate for schools 
and districts, and our education system is about to enter a time of profound fiscal 
austerity. Schools will be pressed to stretch their education dollars further for 
years, perhaps decades. Budget shortfalls have forced more than 30 states to make 
significant reductions in education spending since the recession began. Colorado 
rolled back school spending last year by $260 million, a nearly 5 percent decline 
from the previous year, in cuts amounting to more than $400 per student. 28 
Sagging revenues at the local level have further compounded the problem, with 
more than two-thirds of superintendents reporting firing staff in 2010 and some 
90 percent anticipating having to do so in 201 1. 29 

The budget situation will get worse before it gets better because property-tax valu- 
ations typically lag behind actual property values. The full impact of the housing 
market collapse, therefore, has yet to hit many state and local budgets. Many states 
and districts have also failed to meet rising pension costs, leading to a nearly $500 
billion teacher pension funding shortfall. And the federal stimulus that helped 
many districts get through the worst of the recession will soon dry up. States will 
have $38 billion less in stimulus funds in 2012. By the end of the same year the 
federal funds will essentially be gone. 30 
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"Rather than 
jumping from 
bandwagon to 
bandwagon, 
enough time 


School and districts have a lot riding on their response to the economic crisis. The 
public was showing impatience with the slow pace of school reform even before the 
recession began, and outcomes remain low. Most fourth and eighth graders are not 
performing on grade level in either reading or math. 31 Three out of every 10 students 
fail to finish high school with a diploma. 32 At a time when states are projecting more 
than $ 1 00 billion in budget gaps, educators need to be able to show that education 
dollars produce significant outcomes, or the public might begin to see schools as a 
bad investment. Put differently, if education systems don’t deliver maximum results 
for the dollar, public trust in public education could eventually evaporate. 


is spent on 
initiatives to see 
the results," said 
Donald Beaudette, 
superintendent 
of Norwell 
Public Schools in 
Massachusetts. 


But the economic pressures also offer opportunity, and many forward-looking 
educators have been using the bleak fiscal situation to experiment with more 
efficient educational approaches. Some school systems have begun offering online 
classes, which offer courses in specialized fields often at lower cost. Others have 
rolled back inefficient policies, and the Los Angeles’ board of education recently 
approved a landmark settlement that limits the traditional practice of laying off 
teachers strictly on the basis of seniority. 33 Texas last year released a study looking 
at how schools and districts spent their money to raise student achievement. 34 

A lasting transformation of the nation’s school system will require more than a few 
islands of dedicated reform, however. For our nation to dramatically increase its 
educational productivity, we must fundamentally rethink how we organize, man- 
age, and fund schools. The challenge for reformers, then, is to create smarter, more 
productive school management systems that encourage all educators to seek out 
new ways to improve educational efficiencies and outcomes. 
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Methodology 


Our goal in this project was to measure academic achievement relative to a dis- 
trict’s educational spending, while controlling for factors outside their control, 
such as cost of living and degree of student poverty. Our work builds on a 2007 
report, “Leaders and Laggards,” published by the Center in partnership with the 
U.S. Chamber of Commerce and Frederick Hess of the American Enterprise 
Institute. In that study, we evaluated state-level returns on investment, comparing 
scores on the National Assessment of Educational Progress with a state’s educa- 
tion spending after controlling for differences in student poverty, special educa- 
tion enrollments, and cost of living. 

Our measures build on the excellent work of many other researchers. Standard 
& Poor’s School Evaluation Services produced in recent years a district 
return-on-spending index. 35 It looked at the percentage of students achieving 
proficiency in reading and math for every $ 1,000 spent per student on core 
operations. Florida conducts an annual productivity examination for each of 
its schools, and the state uses a methodology similar to our Basic Return on 
Investment Index, described below, which compares school-level reading and 
math gains against adjusted expenditures. 

Our approach was aided by an advisory group that included Bruce Baker, an asso- 
ciate professor at the Graduate School of Education at Rutgers University; Gary 
Bass, founder and executive director of OMB Watch; Jack Buckley, an associate 
professor of applied statistics at New York University (and now the commissioner 
of the National Center for Education Statistics); William Duncombe, a profes- 
sor of public administration and associate director of the Education Finance 
and Accountability Program at Syracuse University; Daria Hall, director ofK-12 
policy development at the Education Trust; Craig Jerald, president of Break the 
Curve Consulting; Raegen Miller, associate director for education research at the 
Center for American Progress; and Marguerite Roza, research associate professor 
at the University of Washington’s College of Education (and now a senior data and 
economic adviser at the Bill & Melinda Gates Foundation). 
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We also solicited the advice of practitioners including Dr. Bonita Coleman-Potter, 
deputy superintendent of Prince George’s County (Maryland) Public Schools; 
productivity experts such as Eric Hanushek, the Paul and Jean Hanna Senior 
Fellow at the Hoover Institution of Stanford University; and education reform 
advocates including Van Schoales, executive director of Education Reform Now, 
a national educational policy advocacy group. Finally, we hired an independent 
researcher to examine our work and ensure that our results were broadly replica- 
ble. Nevertheless, we take full responsibility for the methodology and evaluations. 


Data sources 

We produced productivity evaluations for more than 9,000 districts that enroll 
more than 85 percent of all U.S. students. We were unable to produce results for 
Alaska, the District of Columbia, Hawaii, Montana, and Vermont. Hawaii and 
D.C. are single-district jurisdictions, so within-state comparisons were not pos- 
sible. Montana and Vermont likewise did not have enough comparable districts. 
We excluded Alaska because we could not sufficiently adjust for cost-of-living 
differences within the state. 

Spending data came from the Local Education Agency Finance Survey, also known 
as the F-33, produced by the federal government’s National Center for Education 
Statistics, or NCES, the primary federal entity for collecting and analyzing data 
related to education. These data are from the 2007-08 school year, the most recent 
year for which complete data are available. Since that time, districts may have taken 
steps that might have significantly changed their efficiency ratings. 

We used the “current expenditures” category, which includes salaries, services, 
and supplies. It does not include capital expenses, which tend to have dramatic 
increases from year to year, and thus are unreliable for comparisons. The expen- 
diture data include money from all revenue sources, federal, state, and local. We 
subtracted from this sum any payments to private schools and charter schools in 
other districts to come up with per-pupil expenditures, as is NCES practice. The 
data were downloaded from the NCES website on October 18, 2010. 

We restricted our study to districts with at least 250 students that offered school- 
ing from kindergarten to 12th grade. We also excluded districts classified as a char- 
ter school agency, state- operated institution, regional education services agency, 
supervisory union, or federal agency. 36 Data from New York City Public Schools 
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were also aggregated into a single district. And to ensure that we had a sufficient 
number of comparable districts in each state, we included states only if more than 
50 percent of their students were covered by our analysis. 

We also relied on NCES to calculate district-level demographic data for the 2007- 
OS school year, the number of students receiving free and reduced price lunch, the 
number designated as English language learners, and the number that participate 
in special education. We downloaded this data from the NCES Common Core of 
Data website on October 18, 2010. 

Many districts did not report demographic data for the 2007-08 school year, 
necessitating the use of proxies. If a school district was missing a demographic 
indicator, we substituted data from either the 2008-09, 2006-07, or 2005-06 
school year. Because demographic data can vary over time, we did not use data 
from more than three different school years for any demographic indicator. The 
Common Core of Data did not report the number of students eligible for free 
and reduced-price lunch for a number of large North Carolina districts, and so 
we obtained the data for seven districts in the state — Bertie, Johnston, Robeson, 
Sampson, Union, Vance, and Wake — from 2008 compliance reports. 37 In no 
instance did we use proxies for achievement or expenditure data. A list of districts 
for which we used proxy data from alternate years is available on our website at 
http: / / www.americanprogress.org/ROI. 

Achievement data came from the New America Foundation’s Federal Education 
Budget Project, which collects data from the states on district-level student out- 
comes. We used these data to create an achievement index, developing a score for 
each district by averaging together the percent of students designated proficient or 
above on the state assessment in reading and math in fourth grade, eighth grade, 
and high school for the 2007-08 school year. Because we did not have the total 
number of students who scored proficient or above, we simply averaged together 
the percent proficient for each subject and grade level. 

The Federal Education Budget Project excludes districts characterized by NCES 
as a charter school, state-operated institution, regional education services agency, 
supervisory union, or federal agency. It also does not include any charter districts. 
The budget project also includes only districts created before 2006. We down- 
loaded the data from the project’s website on October 18, 2010. 
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Our three productivity measures 


To emphasize the complexity of measuring a district’s productivity, we offer three 
different approaches to measuring productivity rather than a single ranking. The 
companion website to this report allows the public to compare districts in a state 
using each of our metrics, as well as to easily compare school systems with similar 
demographics and size. The site also details each district’s achievement and spend- 
ing data. We used shades of colors when ranking the districts to emphasize the fact 
that we did not evaluate districts against an external benchmark but rather on their 
relative performance. 


Basic Return on Investment index rating 

This measure rates school districts on how much academic achievement they get 
for each dollar spent, relative to other districts in their state. 

Because it costs more to educate certain populations than their peers, we adjusted 
the expenditure data for students in special programs, such as students who 
receive subsidized lunches and are in special education. This is a common practice 
in school finance research, and we derived the weights by calculating the average 
weight used in a half-dozen research studies and policy papers. 38 Based on those 
calculations, we used a weight of 1.4 for free and reduced-price lunch, 1.4 for 
English-language learners, and 2.1 for special education. 

To understand how this works, consider an example. The research indicates that 
each student who qualifies for a subsidized lunch costs about 40 percent more 
to educate. So, for each additional student in the free and reduced-priced lunch 
program, we subtracted 40 percent from the district’s per-student spending. 

To adjust for cost-of-living differences, we used the Comparable Wage Index, a 
measure of regional variations in the salaries of college graduates who are not 
educators. Lori Taylor at Texas A&M University and William Fowler at George 
Mason University developed the CWI to help researchers fine-tune education 
finance data to make better comparisons across geographic areas. We used adjust- 
ments from 2005, the most recent available. 

To calculate the adjusted costs for each district, we created a needs index designed 
to measure how much additional funding a school district should have received 
based on its students in special programs, including the percentage of students 
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in the subsidized school lunch program, special education students, and English- 
language learners. We created the index by multiplying the number of students in 
these special programs by their respective weight. We then divided the weight by 
the enrollment to get the average additional amount of funding that a given school 
district should have received. To avoid penalizing districts with greater needs, we 
then divided the raw per-pupil expenditure by the weighted index to produce the 
amount of money a district would have spent if it had no students in special pro- 
grams. Finally, we adjusted this measure by the CWI to make it comparable across 
different geographic localities. 

We then distributed districts in each state into three equal tiers based on their 
position on the achievement index, with the highest achievers in the top tier and 
the lowest achievers in the bottom tier. We also divided the districts into three 
equal tiers based on their adjusted expenditures, with the highest adjusted spend- 
ers in the top tier and the lowest adjusted spenders in the bottom tier. Then we 
used an evaluation matrix to assign colors to each district based on their achieve- 
ment tier relative to their spending tier, with green being the most productive and 
red being the least productive. 

The matrix rewards districts that had low spending and high achievement relative 
to other districts in their state. So if a district was in the top third of achievement 
and the bottom third in spending, it would receive a rating of green. 


ROI Evaluation Matrix 


Lowest achievement Medium achievement Highest achievement 



Lowest cost 
Medium cost 


To understand better how our Basic ROI Index works in practice, consider 
Maryland. We first ranked the state’s 24 districts along our achievement index. 
That put districts with relatively high achievement, such as Queen Anne’s County 
Public Schools, in the top achievement tier. (Queen Anne’s County has an 
achievement index of 87 and ranks sixth in the state on that measure). Districts 
with relatively low achievement, such as Dorchester County Public Schools, went 
into the bottom achievement tier. (Dorchester County has an achievement index 
of 72 and ranks third from the bottom on this measure.) 
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Then we looked at each district’s adjusted spending. Dorchester County had 
high adjusted spending, and so it went into the highest adjusted spending tier. 
(Dorchester’s adjusted per-student spending is $10,462 and ranks 17th out of 24 
districts on this measure.) Queen Anne’s County had relatively low adjusted spend- 
ing, and so it went into the lowest adjusted spending tier. (Queen Anne’s adjusted 
per-student spending is $8,648 and ranks seventh in the state on this measure.) 

Then we used the evaluation matrix (see box, page 19) to assign colors to each 
district based on its achievement tier relative to its spending tier. Queen Anne’s 
County had high achievement and low adjusted spending, and so it received a 
green rating. Dorchester had low achievement and high adjusted spending, and so 
it received a red rating. 


Adjusted Return on Investment index rating 

This measure uses the same approach as the Basic ROI rating but applies a differ- 
ent statistical method, called a regression analysis, to account for factors outside a 
district’s control, such as the added costs of educating low-income, non-English- 
speaking, and special education students. The adjustments, or weights, used in 
the Basic ROI are not always sensitive enough to account for spending differences 
within states. For example, states might provide districts with additional funding 
for students in special education, and thus a weight of 2.1 for a student in special 
education might be too high. 

In this approach we predicted what a district would spend relative to other 
districts in the state. We ran the regression models separately for each state to 
account for variation within each state’s educational financing system. Here’s the 
process depicted as an equation: 

ln(CWI adjusted ppe)= (5 0 + $°/o free lunch + P 2 % ELL+ |i 3 % Special Ed + e 

We predicted each district’s spending based on the percentage of students in spe- 
cial programs, including the percentage of students receiving free or reduced-price 
lunch, the percentage designated as English-language learners, and the percentage 
who participate in special education. Thus, we predicted how much more or less 
the school district is spending than what we predicted it should be spending — 
also known as a residual — and we used this as our measure of spending. 
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We then divided the districts into three tiers based on how much more or less the 
district spent than what we predicted it should have spent. Districts with lower- 
than-predicted scores went into the lowest tiers, and those with higher-than-pre- 
dicted scores into the highest tier. 

We then used the achievement index to separate the districts into three tiers, as in 
the Basic ROI rating. Finally, we assigned each district a color on the evaluation 
matrix based on its placement on the achievement and predicted-spending tiers. 

To get a sense of how this worked in practice, consider again the Maryland 
example. First, we ranked all the districts in the state based on their achievement 
indexes. Again, districts with high achievement, such as Queen Anne’s County, 
went into the top tier while relatively low- achieving districts, such as Dorchester 
County, went into the bottom tier. 

Next, we looked at each district’s predicted spending score, or the difference 
between the predicted value and the actual value. Dorchester County had an 
average predicted spending score, or residual, and so it went into the middle tier 
for predicted spending. Queen Anne’s County had an average predicted spending 
score, or residual, and so it went into the middle tier for predicted spending. 

Then we compared the districts against our evaluation matrix. Dorchester 
County had low achievement and a middling predicted spending score, or 
residual, and so it received a rating of dark orange. Queen Anne’s County had 
high achievement and a middling predicted spending score, or residual, and so it 
received a rating of light green. 


Predicted Efficiency index rating 

This measure is significantly different than the first two measures. 

The first two measures rate districts based on the achievement that school systems 
produce compared to their expenditures after controlling for factors outside the 
district’s control. In contrast, the predicted efficiency measure doesn’t compare 
achievement to spending. Instead, the approach rates districts on the results of 
their predicted achievement after controlling for factors outside their control. This 
distinction is important. The first two approaches attempt to measure how much 
“bang for the buck” a school district gets. This third approach attempts to elimi- 
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nate the effects of spending and other factors such as students with additional 
needs and then evaluates districts by how much more or less achievement the 
district produced than would be expected. 

Technically then, this approach does not evaluate districts against an evaluation 
matrix, nor does it weight or predict the amount that a school district spends on 
education. Instead, we used a regression analysis to predict what achievement a 
district should have relative to other districts in the state given its spending and 
percentage of students in special programs. 

To calculate this estimate, we used a production function, a type of regression 
analysis that examines the relationship of inputs to an output, and we predicted 
the achievement index as a function of the district’s cost of living adjusted 
per-pupil expenditure, the percentage of students participating in the free and 
reduced-priced lunch program, the percentage of students who are English- 
language learners, and the percentage of special education students. 

This approach is shown in equation form below: 

achievement = (3 0 + ln(CWI adjusted ppe) + (3, % free lunch + (? 3 % ELL + % 

Special Ed + e 

To control for differences in state finance systems, we calculated individualized 
production functions for each state. Then, after predicting each district’s achieve- 
ment, we divided the results into six bands and awarded colors to districts that 
produced higher or lower levels of achievement than would be expected, with 
green being the most productive and red being the least productive. Districts with 
negative scores — or those that produced a lower level of achievement than would 
be expected — were given the least desirable rankings. 

One of the limitations of the Predicted Efficiency index is that districts with high 
overall achievement can receive low productivity scores. That is not the case with 
the first two productivity approaches. The measure also adjusts academic expec- 
tations for students from disadvantaged backgrounds. While this is an accepted 
research practice in the education policy community, the Center for American 
Progress opposes the lowering of academic expectations as a matter of policy. 

The reasons are both philosophical and practical. Philosophical because we do 
not believe that a country that promises that everyone is created equal should 
have lower educational standards for students who are from low-income families 
or speak English as a second language. Practical because we believe that unless 
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schools have high academic expectations, we will not ensure that all students — 
regardless of family background — will succeed. But as we researched various 
productivity measures, we found that this approach provided important insight 
into a district’s productivity and helped provide a more well-rounded understand- 
ing of its overall efficiency. 

Consider the districts in Maryland again as an example. On the Predicted 
Efficiency index, Queen Anne’s County received just below-average marks, and 
it earned a rating of orange. That means that it did less well relative to other 
districts than would be expected, given its spending and percentage of students 
in special programs. To help understand that result, consider that Queen Anne’s 
has 15 percent of its students in the subsidized school lunch program, and in 
Maryland, the percent of students who participate in the program have a large 
and negative impact on achievement, one of the largest of any of the variables 
included in our regression. 

The regression model predicted Queen Anne’s achievement relative to other 
districts, and there are districts in the state that have similar demographics that 
spent less and achieved more, which helps to explain why Queen Anne’s received 
just below-average marks. Calvert County, for instance, has largely similar percent- 
ages of students in special programs as Queen Anne’s, but it has an achievement 
index score of 90, three points higher than Queen Anne’s, and Calvert’s adjusted 
per-student spending was $8,091, about $500 less per student. 

Dorchester County also received just below-average marks on this metric, and it 
received a rating of orange. This indicates that the district did less well than was 
estimated, given its spending and percentage of students in special programs. 

To help explain the evaluation, consider that Dorchester County enrolls a large 
percentage of low-income students, with about half of the students in the district 
participating in the subsidized school lunch program. Again, the percentage 
of students who receive subsidized lunches in Maryland has a large and nega- 
tive impact on predicted achievement, one of the largest of any of the variables 
included in our study. 

Our regression model compared Dorchester County’s achievement to that of 
other districts, taking into account the low performance of students who partici- 
pate in the subsidized lunch program. Dorchester’s results were just below average 
in part because there are districts in the state with similar rates of poverty, such as 
Allegany County Schools, that have significantly higher achievement. 
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Important caveats 


One of the aims of our study is to draw attention to the large variance in produc- 
tivity within states, and while we believe that our district-level evaluations rely on 
the best available methods — and show important and meaningful results — we 
caution against making firm conclusions about the ratings of an individual district. 

The literature on productivity is limited, and there’s a lot we don’t know about the 
relationship between spending and achievement. It appears, for instance, that the 
link between outcomes and money is not always linear. In other words, even in an 
efficient school system, the first few dollars spent on a program or school might 
not have the same effect as subsequent expenditures, with additional dollars not 
boosting outcomes as much as initial investments. We also know that additional 
resources are often provided to districts that already have high achievement and 
that this can potentially mask inefficiencies in spending. 

Because of the limitations of the research, we could not evaluate the efficiency 
of a district against an external benchmark. We therefore rated districts based on 
their relative performances. That means a few things. First, we slotted districts 
into different evaluation levels even though in some cases the numerical value that 
separated the districts may not have been significant. It also means that states with 
a smaller number of districts had different cutoff points between rating categories 
than did states with larger numbers of districts. 

Our measures also cannot account for all of the variables outside the control of 
a district, in large part because the field of education suffers from a lack of high- 
quality data. School-by-school spending data, for instance, are not available in 
most states. That’s why we were able to produce only district-level productivity 
results, which likely mask significant variation within a district. And apart from 
excluding any district serving fewer than 250 students, we did not adjust for econ- 
omies of scale. There are issues with the data as well as debate within the research 
community about what economies of scale say about the quality of a district’s 
management. 39 But given the potential impact that size can have on spending, we 
made it easy to sort by both enrollment and geography on our interactive website 
so that users can compare similar districts. 

The available data are also problematic. State and district data often suffer from 
weak definitions and questionable reliability. For instance, the federal government 
requires that every school report the number of students who participate in the 
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free and reduced-price lunch program. But schools rely on parental self-reporting 
to determine eligibility and so schools that are more aggressive about recruiting 
families into the program often have higher participation rates, even though they 
might not necessarily have larger percentages of low-income students. 

Other data released by NCES appear to be simply flawed. Take, for instance, 
Connecticut’s New Canaan Public Schools, which is located about an hour north 
of New York City. In 2008, NCES reported that close to 100 percent of New 
Canaan’s students received free and reduced-priced lunch. That would make 
New Canaan one of the poorest districts in the country. But only 2.2 percent 
of students in the New Canaan area are poor, according to the Census Bureau. 
That’s well below the state average of 1 0 percent. 40 When we informed N CES of 
the contradiction, an official said the state had reported the data to them and that 
there was no way for them to verify if the figure was too large or small. (We used 
subsidized lunch as the measure of poverty for our evaluations because it’s the 
only poverty indicator available at the district level. The rest of the poverty data 
are available only at the county or municipal level.) 

There are problems with achievement data, too. Many state assessments don’t 
rigorously assess what students know and are able to do. Some of the exams use 
only multiple-choice questions to test student mastery of a subject, thus providing 
limited perspective on student skills. Other exams are not properly aligned with 
state curriculum standards and maybe too easy. Moreover, our study looks only at 
reading and math test scores, an admittedly narrow slice of what students need to 
know to succeed in college and the workplace. 

Despite these caveats, we believe our evaluations are useful, and the best available, 
given existing traditions and knowledge. We designed our color-rating system to 
empower the public to engage the issue of educational productivity, and we’ve 
produced an interactive website that allows users to compare the productivity of 
similar districts. We hope this project promotes not just further talk and deeper 
research — but also thoughtful action to maximize school spending. 
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Findings 


This report is designed to spark a national conversation about educational produc- 
tivity and to identify districts that generated more relative achievement per dollar 
spent. Below are our major findings: 


Inefficient school systems represent a significant reform 
opportunity 

If school systems spent their dollars more productively many would see large 
gains in student achievement. Consider California, where a low-productivity 
school district could see as much as a 25 percent boost in achievement if it 
increased its efficiency from the lowest level to the highest, all else being equal. In 
Arizona, that jump in achievement could be more than 36 percent, according to 
our analysis. 

Our data show that 41 states show the potential for double-digit percentage 
increases in achievement without necessarily spending additional funds. Such 
growth in student learning will not come without significant reform since the pro- 
grams and policies that cause low productivity are often systemic. But at a time of 
sagging revenues and pending budget cuts, these results should inspire states and 
districts to tackle productivity head-on and consider reforms that boost achieve- 
ment without incurring significant costs. (See Table A2) 


Low productivity is costing the nation's school system as much as 
$175 billion a year 

After adjusting for variables outside a district’s control, districts with below- 
average productivity spent over $950 more per student than did above-average 
districts. This estimated loss in capacity equals about 1 percent of the nation’s 
growth domestic product. 


26 Center for American Progress | Return on Educational Investment 


To be sure, inefficient districts are not necessarily “wasting” the lost capacity. Our 
approach cannot account for all the factors outside of a district’s control, and the 
extra money spent by some districts might be supporting outcomes beyond the 
scope of this study. But at the same time, our estimate might also be low since it 
does not cover the cost of poorly prepared students entering college and the work- 
force. Far more research needs to be done in this area in order to better under- 
stand the scope of the productivity problem. 


Without clear controls on how additional school dollars are spent, 
more education spending will not automatically improve student 
outcomes 

Additional dollars corresponded to higher student achievement in only 16 states. 
In five states, including Florida, Texas, and New Jersey, additional dollars pre- 
dicted slightly lower achievement. 

That does not mean that money can’t — or doesn’t — have an effect on achieve- 
ment in Florida, Texas, or anywhere else. Rather, it means that money matters 
only if it’s spent in effective ways, at least above some threshold level, and that 
without a systemic approach to spending, unfocused increases in expenditures are 
not likely to have any impact on student outcomes. 


Efficiency varies widely within states 

Some districts spent thousands more per student to obtain the same broad level of 
academic achievement. In New York, the range of spending among the districts in 
the highest third of achievers was more than $7,000 per student. In California, the 
range was almost $8,000 per student. 

The differences are starker when comparing similar districts, like Oshkosh and 
Eau Claire, Wisconsin. The two school systems are about the same size, have 
largely comparable results on state exams, and serve similar demographics. But in 
2008, Eau Claire spent over $8 million more than Oshkosh, or about $300 more 
per student. (See page 32) 
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More than 1 million students are enrolled in highly inefficient 
districts that received a low rating on all three of our measures 

That’s over 400 school districts, serving about 3 percent of the almost 43 million 
students covered by our study. These districts spent far more money than other 
districts, and after adjusting for characteristics outside the district’s control, these 
low-productivity districts spent over $2,000 more per student than the average 
school district. 


High-spending school systems are often inefficient 

Our analysis showed that after accounting for factors outside a district’s control, 
many high-spending districts posted limited outcomes. In Minnesota just 23 
percent of the districts in the top third of spending were also among the top third 
in achievement. In Florida, only 17 percent of the state’s highest- spending districts 
were also in the highest-achieving tier. 

In high-spending districts, success often comes at significant cost. Consider 
Howard County School District in Maryland. Many consider the affluent district 
to be one of the best in the country, and a magazine recently heralded the district 
as an international academic “powerhouse.” 41 But after controlling for factors 
outside the district’s control, the school system has one of the highest rates of per- 
student expenditures in the state, spending over $1,600 more per student than the 
state average and almost $3,000 more than the national average. 

To be sure, our evaluation could not control for all the variables that go into mea- 
suring an efficient school district, and some of the money expended by high-spend- 
ing districts like Howard Country may support high-quality programs and tools 
such as science labs and computer rooms that would not be captured in math and 
reading tests. But given the need to dramatically improve the nation’s reading and 
math outcomes, taxpayers and parents should scrutinize high-spending districts 
and ensure that they’re getting everything that they can for their school dollar. 
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Students from disadvantaged backgrounds are more likely to be 
enrolled in inefficient districts 


Our data showed that students who participated in the subsidized-lunch program 
were 12 percentage points more likely to be enrolled in the nations least-productive 
districts than the most productive. This finding appears after adjusting expenditures 
for the higher cost of educating lower-income students and suggests that highly inef- 
ficient districts are more likely to have larger percentages of students from disadvan- 
taged backgrounds. 

Students from minority backgrounds are also more likely to be enrolled in highly 
inefficient districts. The least- efficient districts were more likely to have larger 
percentages of black students (18 percent versus 5 percent) and Hispanic students 
(14 percent versus 7 percent) than the most efficient ones. But while some within 
the research community use race as an adjustment within regression models to 
account for the fact that students of color often perform worse than their peers, we 
did not do so for a number of reasons. Race variables, for one, did not add signifi- 
cantly to the robustness of our productivity evaluations. The Center also does not 
support policies that lower academic expectations for students of color or from 
disadvantaged backgrounds. 

These findings present an important avenue for additional research. Our data cannot 
capture everything that goes into creating an efficient school system, and there were 
confounding variables that our study was not able to control for. For instance, school 
systems with weak tax bases maybe subject to greater administrative burdens and 
have less control over how funds are spent, which might make it harder for them 
to be efficient. Such school systems also typically tend to have greater proportions 
of low-wealth parents, whose students often perform less well on reading and math 
exams. But we could not account for differences in revenue sources because the fiscal 
database produced by NCES does not track expenditures by source, and this may 
make districts with weaker tax bases appear less efficient than they really are. 


Highly productive districts are focused on improving student 
outcomes 

We surveyed a sample of the districts that performed well on each of our productiv- 
ity metrics, and they shared a number of attributes, such as building support within 
their communities for productivity reforms and using data-mining practices to 
reduce inefficiencies. (See page 36.) 
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States and districts fail to evaluate the productivity of schools and 
districts 

Only two states, Florida and Texas, currently provide annual school-level produc- 
tivity evaluations, which report to the public how well funds are being spent at the 
local level. But without consistent metrics, educators will not be able to figure out 
if school dollars are well spent. 


The quality of education data is often poor, which impedes the 
study of educational productivity 

In far too many cases, crucial data on school finance, operations, and outcomes 
are unavailable, making it difficult to accurately measure the achievement that a 
school district produces relative to its expenditures. For instance, we could not 
control for certain cost factors, such as transportation, due to a lack of robust data, 
and so a rural district that has high busing costs because its students are spread 
out over a large area might be at a disadvantage in some of our metrics relative to a 
more densely populated district. 

When states and districts do collect key education data, they often use inconsistent 
definitions and weak data collection practices. For instance, policymakers in some 
states set the cut scores of state exams at very low levels so that many school systems 
report having proficiency levels at 90 percent or above. Six districts in Nebraska 
reported that all their students were proficient in reading and math in fourth grade, 
eighth grade, and high school in 2008. This makes it difficult to compare the produc- 
tivity of districts, since some have essentially topped the achievement scale. 


The nation's least-productive districts spend more on 
administration 

The most inefficient districts in the country devote an extra 3 percentage points of 
their budgets on average to administration, operations, and other noninstructional 
expenditures. This translates into large per-student spending differences, and after 
adjusting for students in special programs and cost of living, the least productive dis- 
tricts spend almost $300 more per student than the average district on student and 
staff support, which includes expenditures on school libraries, media centers, and 
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guidance counselors. The least productive districts also spend over $350 more per 
student than the average district in administrative costs, which includes dollars spent 
on central services such as payroll as well as principals and other administrators. 

This finding does not mean that high administrative costs cause low productivity. 
Inefficiencies are often buried deep within the operation of school systems. 42 The 
problem might be large expenses on programs that do little to raise student achieve- 
ment, or salaries paid out to staff that have little or nothing to do with the employee’s 
effectiveness. It’s also possible that our measures reflect the fact that districts with 
lower achievement are often subject to increased state regulations. In that case, it 
wouldn’t be high administrative spending causing low efficiency, but low efficiency 
causing increased administrative burdens. We also do not endorse policy proposals 
that require a set amount of money per student to be spent in the classroom. Such 
blunt formulas often do more to hinder local administrators than help them. 


Some urban districts are far more productive than others 

While our main results are limited to within-state comparisons, we conducted a 
special cross-state analysis of urban districts that recently participated in a national 
achievement test. And after adjusting for certain factors outside a district’s control, 
we found that some big-city school systems spend millions of dollars more than 
others — but get far lower results on math and reading tests. (See page 33) 
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A tale of two districts 


The school districts of Oshkosh and Eau Claire in central Wisconsin 
are remarkably similar. Both have about 10,000 students. Both serve 
similar demographics, with about 30 percent of students qualifying 
for free and reduced-priced lunch. They also have largely comparable 
test results, and 83 percent of fourth grade students in both districts 
scored proficient or above in reading in 2008. They both even sent a 
high school basketball team to the state finals in 2009. 

But Eau Claire spent about $330 more in unadjusted dollars per 
student to run its school system, or more than $8 million in 2008. 

That difference registered on our productivity metrics. Oshkosh 
received a light green evaluation on each of our three return-on- 
investment measures, while Eau Claire scored a yellow on each of 
the three metrics. 

When we examined the 2008 budgets of the two districts, we found 
that Oshkosh spent 65 percent of its budget on instructional expendi- 
tures, while Eau Claire dedicated just 59 percent of its money to costs 

TABLE 1 

Total current expenditures for Eau Claire School District, 
Wisconsin 


associated with teaching. Eau Claire, meanwhile, outspent its sister 
district 1 6 percent to 1 1 percent on administrative expenditures as a 
share of total budgets. These different priorities have a large impact 
on the districts' budgets. In 2008 unadjusted dollars, Eau Claire spent 
almost $7 million — or almost $600 more per student — in administra- 
tive expenditures than Oshkosh. 

To be sure, Eau Claire may be getting something valuable for its ad- 
ditional spending. While Oshkosh posted higher eighth-grade math 
scores, Eau Claire showed better high school reading outcomes, and 
Eau Claire scored one point higher on our achievement index, getting 
a score of 81 , while Oshkosh averaged an 80. (The index range in the 
state was from 35 to 95 points, with an average of 81 .) Moreover, our 
study doesn't cover a full range of potential school outcomes, such 
as science achievement or graduation rates. But still the Oshkosh 
comparison suggests that when it comes to recent reading and math 
exams, Eau Claire is spending more to get largely similar outcomes — 
and taxpayers should be asking why. 

TABLE 2 

Total current expenditures for Oshkosh School 
District, Wisconsin 


Spending 

Amount 

Amount per 
student 

Percent 

Total current expenditures 

$111,278,000 

$10,310 


Instructional expenditures 

$66,023,000 

$6,117 

59% 

Student and staff support 

$8,539,000 

$791 

8% 

Administration 

$18,126,000 

$1,679 

16% 

Operations, food service, other 

$18,590,000 

$1,722 

17% 


Source: U.S. Department of Education, National Center for Education Statistics, Current Expenditures for 
Public Elementary and Secondary Education: School Year 2007-08. 


Spending 

Amount 

Amount per 
Student 

Percent 

Total current expenditures 

$103,083,000 

$9,979 


Instructional expenditures 

$66,530,000 

$6,440 

65% 

Student and staff support 

$9,271,000 

$897 

9% 

Administration 

$11,341,000 

$1,098 

11% 

Operations, food service, other 

$15,941,000 

$1,543 

15% 


Source: U.S. Department of Education, National Center for Education Statistics, Current 
Expenditures for Public Elementary and Secondary Education: School Year 2007-08. 
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Wide variations in urban district 
productivity 


Because each state has its own student assessment program, the return-on-invest- 
ment measures used in this report are restricted to within-state comparisons of 
school districts. 

We were able, however, to conduct a special cross-state analysis of the urban 
districts that participated in the Trial Urban District Assessment, or TUDA. That 
trial program uses the National Assessment of Educational Progress test, known as 
the “nation’s report card,” the only source of comparable student performance data 
across states. Because of the differences in methodologies, the TUDA data is not 
always directly comparable to the achievement results reported in the companion 
website to this report. 

What we found is that some large urban school systems get more bang for their 
buck than others. After adjusting for certain factors outside a district’s control, 
such as cost of living and student poverty, some big-city school systems spend mil- 
lions of dollars more than others — but get far lower results on national math and 
reading exams. 

Consider the Austin Independent School District in Texas, one of the highest- 
scoring districts on the most recent TUDA exam. After adjusting for factors such 
as cost of living and students in poverty, Austin spent $6,450 per student in 2008. 
That’s about $2,500 less per student in adjusted costs than what was spent by the 
Baltimore City Public School System, where the percentage of students scoring 
at or above proficient is lower by at least 20 percentage points in each of the areas 
assessed by the exam. 43 

Austin and Baltimore are far from perfectly comparable school systems. While 
the two districts enroll similar numbers of students (around 80,000 students), 
Baltimore had a higher percentage of students receiving subsidized lunches (73 
percent to 61 percent) and special education students ( 17 to 10 percent) in 2008. 
But Austin has a greater proportion of students who are English language learners 
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(13 percent to 2 percent) . And while we controlled for poverty and students in 
special education, we could not fine-tune our adjustments to control for differ- 
ences in the type of poverty or disability Nor could we adjust for all of the other 
factors outside of a district’s control. This means that differences in achievement 
and expenditures do not necessarily indicate that districts like Baltimore are 
squandering the additional dollars they spend. But the data suggests that when it 
comes to fourth and eighth grade reading and math scores, the public should be 
asking some districts if their education dollars could have been spent differently to 
produce higher results. 

For their part, the nations urban school districts have been working to improve 
their productivity. The Council of Great City Schools, a coalition of 65 of the 
nation’s largest public school systems, recently announced an initiative to boost 
efficiency. The Managing for Results in America’s Great City Schools project pro- 
vides districts with specific spending and operations targets so that school systems 
can better identity potential cost-savings. For example, consider a district that had 
100 buses and only 69 percent were in operation on a given day. Since the percent- 
age of buses in operation was well below the median for urban schools, the district 
might use the benchmarked data to argue that it should sell 16 of its fleet and 
save $320,000, or enough for five new teachers. 44 Such changes will not happen 
overnight, but they hold the promise of saving taxpayers millions of much-needed 
school dollars. 
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TABLE 3 

Urban district achievement and spending and demographic indicators 


District 

Percentage of 
4th graders 
scoring at or 
above the 
proficient level 
on NAEPTUDA 
math exam 

Percentage of 
4th graders 
scoring at or 
above the 
proficient level 
on NAEPTUDA 
reading exam 

Percentage of 
8th graders 
scoring at or 
above the 
proficient level 
on NAEPTUDA 
math exam 

Percentage of 
8th graders 
scoring at or 
above the 
proficient level 
on NAEPTUDA 
reading exam 

Percentage of 
all students 
in the district 
who are low 
income 

Percentage of 
all students 
in the district 
who are in 
special educa- 
tion 

Percentage of 
all students 
in the district 
who are Eng- 
lish language 
learners 

District per- 
pupil spend- 
ing, adjusted 
for student 
needs and cost 
of living 

Enrollment 

Atlanta 

21% 

22% 

11% 

17% 

76% 

9% 

3% 

$9,224 

49,991 

Austin 

38% 

32% 

39% 

30% 

61% 

10% 

13% 

$6,450 

82,564 

Baltimore City 

13% 

12% 

10% 

10% 

73% 

17% 

2% 

$9,124 

81,284 

Boston 

31% 

24% 

31% 

23% 

71% 

21% 

19% 

$11,270 

56,168 

Charlotte 

45% 

36% 

33% 

28% 

43% 

11% 

14% 

$5,805 

131,176 

Chicago 

18% 

16% 

15% 

17% 

76% 

13% 

18% 

$6,245 

407,510 

Cleveland 

8% 

8% 

8% 

10% 

55% 

20% 

5% 

$8,698 

52,954 

Detroit 

3% 

5% 

4% 

7% 

72% 

15% 

7% 

$7,681 

107,874 

Fresno 

14% 

12% 

15% 

12% 

79% 

10% 

26% 

$6,661 

76,460 

Flouston 

30% 

19% 

24% 

18% 

62% 

9% 

14% 

$5,722 

199,534 

Jefferson 

County, 

Kentucky 

31% 

30% 

22% 

26% 

55% 

14% 

5% 

$7,626 

95,871 

Los Angeles 

19% 

13% 

13% 

15% 

68% 

12% 

32% 

$6,856 

693,680 

Miami-Dade 

33% 

31% 

22% 

28% 

59% 

11% 

15% 

$7,405 

348,128 

Milwaukee 

15% 

12% 

7% 

12% 

77% 

18% 

9% 

$8,047 

86,819 

New York City 

35% 

29% 

26% 

21% 

71% 

18% 

15% 

$9,434 

968,722 

Philadelphia 

16% 

11% 

17% 

15% 

68% 

15% 

7% 

$6,072 

1 72,704 

San Diego 

36% 

29% 

32% 

25% 

62% 

13% 

29% 

$6,633 

131,577 


Source: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress, Trial Urban District Assessment, 2009; U.S. Department of Education, 
National Center for Education Statistics, Common Core of Data, 2008; U.S. Department of Education, National Center for Education Statistics, Current Expenditures for Public Elementary and Secondary 
Education, 2008. 


Wide variations in urban district productivity | www.americanprogress.org 35 


A closer look at the most 
productive school districts 


What should schools and districts do to improve productivity? What programs 
and practices do the most productive districts have in common? To answer these 
questions, we analyzed the more than 240 districts that scored well on each 
of our productivity measures. We examined both their achievement and their 
expenditures as well as surveyed a sample of districts to learn more about their 
principles and practices. 45 

The most productive districts were generally larger and more privileged than 
the inefficient districts. They enrolled lower percentages of low-income students 
than the least productive districts (37 percent to 49 percent). They were also 
more likely to be classified as large suburban school systems (20 percent versus 
10 percent) and less likely to have small student bodies (77 percent of the least 
productive districts enrolled less than 2,000 students, in contrast to 51 percent of 
the most productive districts). 

But highly productive districts do vary widely in size, location, and demograph- 
ics. The Hawthorne School District, for instance, a few miles west of Compton, 
California, has a student body that’s more than 85 percent low-income and almost 
40 percent English-language learners, and earned very high marks on each of our 
efficiency metrics. Other highly productive districts were very small and rural, such 
as the 700-student Dolores School District in the mountains of western Colorado. 

For all their diversity, we found that the highly productive districts shared a num- 
ber of attributes. Let’s examine each of them in turn. 


A focus on outcomes 

Highly productive districts reported a laser-like focus on student performance. 
“The biggest driving force [here] is first and foremost the question: ‘How will this 
enhance learning?”’ said Michele Campbell, superintendent of Pennsylvania’s Fort 
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Leboeuf School District. “Expenditures need to fit into our vision and overarching 
educational objectives.” 

The districts used a variety of ways to increase student achievement. Some empha- 
sized low-cost strategies^ such as requiring principals to visit every classroom each 
week to give feedback on instruction. Some tried to create a more collaborative 
teaching culture. Waverly-Shell Rock Community Schools in Iowa has been build- 
ing “learning communities” of teachers to ensure student learning is taking place 
and help educators develop their curricula. 


Strong community relations 

Many of the highly productive districts worked closely with their com- 
munities to help maximize education spending. Franklin Public Schools in 
Massachusetts, for example, merged its technology department with that of the 
town in order to reduce costs. Poyen School District in Arkansas developed an 
agreement with a local community college to offer Poyen students free college 
and vocational courses. 

Strong community relationships can also help with tough fiscal decisions. In 2009, 
St. Lucie County Schools in Florida had to cut $30 million from its budget. The 
union and the administration agreed on a number of cost-saving strategies, includ- 
ing pay freezes so that classroom teachers would be protected as much as possible. 
“There is a unity from our school board, our district and school leaders, and our 
union partners that provide the context of how work is done,” said Kathy McGinn, 
the district’s assistant superintendent for strategic planning and central services. 


A willingness to make tough choices 

Reducing spending while maintaining strong outcomes takes fiscal acumen, 
political savvy, and a willingness to make hard choices. In Taylorville, Illinois, the 
district recently closed two underenrolled elementary schools and laid off admin- 
istrators. The cuts saved the 3,000-student district more than $1 million, and 
the system continues to post solid academic results — with test scores well above 
the state average. “Our communities are getting a good bang for their buck,” said 
superintendent Gregg Fuerstenau. 
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Reducing costs without cutting into achievement also requires careful planning. 
Norwell Public Schools in Massachusetts is in the third year of a comprehensive 
reform plan. A 25-member team of teachers, parents, administrators, and com- 
munity leaders developed the initiative, which was aimed at boosting student 
outcomes and improving efficiency. “Rather than jumping from bandwagon to 
bandwagon, enough time is spent on initiatives to see the results,” said Norwell 
Superintendent Donald Beaudette. “Doing more with less has become a necessity.” 


A priority on quality instruction 

The country’s highly productive districts devoted 3 percentage points more of 
their budget to instructional costs than did the least efficient districts. They spent 
about 61 percent of their dollars on instructional expenditures, which includes 
teacher salaries, curriculum materials, and other classroom costs. “We aggressively 
protect resource [s] ... for direct instructional services,” said Vic Noel Adkison, 
superintendent of Thomasville City Schools in Alabama. 

Highly productive school systems sought large pools of teacher applicants by 
broadening job postings to local newspapers, state employment websites, and area 
universities. They also often employed instructional coaches to help mid-career 
educators hone their skills. “Our teaching staff continues to grow stronger and 
more effective as a result of a careful selection process and the support and train- 
ing provided by the district,” said David Johnson, superintendent of the Harlan 
Independent School District in Kentucky. 


Smart use of data 

Most of the highly productive districts reported having sophisticated data systems 
that provided detailed information on a variety of school outcomes, from parent 
satisfaction to student success in college. Districts reported that the data allowed 
them to understand what works and what doesn’t. “We need to see the possibili- 
ties. When there is an outlier in the district — an upward outlier — that means the 
bar is raised and others can do it,” said Susan Parks, superintendent of San Gabriel 
Unified School District near Los Angeles. 
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Robust data systems also allowed the districts to identify potential cost inef- 
ficiencies. By analyzing its facilities spending, Rio Rancho Public Schools in 
New Mexico was able to post a 23 percent decrease in natural gas usage and a 71 
percent reduction in irrigation water usage. 

In many ways, of course, these highly efficient districts were also a lot like other, 
less productive districts, and they were concerned that additional budget cuts could 
impair future outcomes. Florence District Five in South Carolina, for instance, 
has weathered major revenue reductions over the last three years. “This year we 
received funding levels that reflect spending 10 years ago,” said Superintendent 
John Morris. In trying to protect key academic programs from budget rollbacks, 
Florence has cut everything from custodial uniforms to cell phones. Morris said he 
was worried that if budget reductions continued, his district would not be able to 
continue to post high student outcomes. “Things are getting desperate.” 
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Recommendations 


Our analysis leads us to the following recommendations. 


Policymakers should promote educational efficiency 

We must know far more about how well school systems are investing federal, state, 
and local taxpayer resources, and we hope this report launches a thoughtful con- 
versation about educational efficiency. Specifically, policymakers should: 

• Work with state and federal governments to spark a much-needed dialogue 
about ways for education systems to do more with less 

• Direct academic studies at the state and local level of alternative measures of 
educational efficiency 

• Convene a national advisory panel to make recommendations on how state and 
federal governments can better support local efforts to become more productive 


States and districts must reform school management systems 

Successful organizations reward success, encourage innovation, and ensure the 
efficient use of funds. But these practices are frequently absent in our nations 
education system. Specifically, policymakers should: 

• Create performance-focused management systems that are flexible on inputs 
and strict on outcomes 

• Hold superintendents and principals accountable for the productivity of their 
organizations through the public reporting of efficiency metrics. Currently, only 
two states, Florida and Texas, produce school-level productivity measures 

• Increase the authority that principals and superintendents have over budgets, 
employees, and other operational decisions. For instance, states should elimi- 
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nate mandatory salary schedules, which establish salaries at the state level, 
preventing local districts from setting teacher compensation levels 
• Provide educators with the tools, technology, and training that they need to suc- 
ceed. Among other things, states should offer school administrators strategies 
on how to thoughtfully stretch their school dollar 


Education leaders should encourage smarter, fairer approaches to 
school funding, such as student-based funding policies 

Education policymakers should develop funding policies that direct money to stu- 
dents based on their needs, so that all schools and districts have an equal opportu- 
nity to succeed. Specifically, policymakers should: 

• Link increases in funding to improved student achievement 

• Reduce unnecessary regulations, including the reliance on state categorical pro- 
grams, which often come with unnecessary strings and red tape 

• Develop funding policies where money follows students based on their needs, 
so that all schools and districts have an equal opportunity to succeed 

• Take steps to improve fiscal equity across schools, districts, and states and 
ensure that all students have an equal opportunity to achieve high standards 

• Support competitive funding programs that create opportunities for reform 
and innovation 


States and districts should report far better data on school 
performance 

States and districts should develop data systems that report reliable, high-quality 
information on school finance, operations, and outcomes. Specifically, policy- 
makers should: 

• Advocate for greater transparency surrounding spending data. The federal 
government should also make permanent the reporting of school-level expen- 
ditures, which was mandated as a one-time requirement under the American 
Recovery and Reinvestment Act of 2009 

• Develop statewide data systems that offer accurate collection, analysis, and 
use of high-quality data to track student achievement and other aspects of 
school performance 
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• Require the federal government to annually publish a database that includes 
school-level achievement and accountability data, which is already collected 
through programs such as Title I 

• Collect, process, and report educational data in a timely manner. Currently, 
expenditure data is released at the federal level more than a year after the previous 
school year has ended 
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Appendix A: Select tables and figures 


Table At provides correlations between our measures and select spending and 
demographic indicators. Note that after adjusting for students from low-income 
families, there remains a moderate correlation with the Basic and Adjusted Return 
on Investment indexes and students who receive free and reduced-priced lunch. 
This most likely has a mix of causes, and it highlights the significance of the 
Predicated Efficiency index, which has almost no correlation with poverty. 


TABLE A1 

Correlations between productivity models and key spending and demographic indicators 



Basic ROI 

Adjusted ROI 

Predicted index 

Per-pupil expenditures 

Percent free and reduced- Percent special educa- 
price lunch students tion students 

Basic ROI 

1 

- 

- 

- 

- 

Adjusted ROI 

0.9554 

1 

- 

- 

- 

Predicted ROI 

0.4656 

0.4651 

1 

- 

- 

Per-pupil expenditures 

0.252 

0.2444 

-0.0017 

1 

- 

Percent free and reduced-priced 
lunch students 

0.3953 

0.3934 

0.0082 

-0.1263 

1 

Percent special education students 

0.1772 

0.1572 

-0.0075 

0.2032 

0.1003 1 
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Table A2 offers some descriptive statistics on select indicators that were men- 
tioned in the major findings section. Results of other analysis, including outcomes 
from alternate specifications and models, are available upon request. 


TABLE A2 

Select state spending and productivity indicators 


Range of adjusted Coefficients and R-squared for Achievement 

spending Predicted Efficiency index gains*** 



Between all 
districts in 
the state* 

Between 
all top 
achievers** 

Free and reduced 
lunch 

English-language 

learners 

Special education 

Per-pupil 

expenditure 

R- 

Percent achievement 
increase if a district 
moves from least to 
most productive, all 
else equal 


Coef- 

ficient 

Standard 

error 

Coef- 

ficient 

Standard 

error 

Coef- 

ficient 

Standard 

error 

Coef- 

ficient 

Standard 

error 

squared 

Alabama 

$3,180 

$3,896 

-0.32 

0.03 

0.18 

0.11 

-0.52 

0.23 

0.11 

0.04 

0.67 

11% 

Arizona 

4,635 

3,659 

-0.21 

0.06 

-0.34 

0.12 

-0.33 

0.51 

-0.19 

0.07 

0.56 

36 

Arkansas 

2,752 

2,201 

-0.47 

0.04 

0.19 

0.10 

0.01 

0.20 

-0.04 

0.05 

0.50 

25 

California 

6,175 

7,936 

-0.51 

0.03 

-0.02 

0.04 

-0.43 

0.19 

0.04 

0.02 

0.70 

25 

Colorado 

4,723 

5,117 

-0.36 

0.04 

-0.35 

0.08 

-0.04 

0.25 

0.13 

0.04 

0.58 

23 

Connecticut 

5,081 

4,807 

-0.29 

0.04 

-1.00 

0.17 

-0.85 

0.28 

-0.03 

0.04 

0.80 

10 

Delaware 

5,513 

2,518 

-0.61 

0.28 

0.57 

0.74 

0.47 

0.77 

0.10 

0.15 

0.40 

18 

Florida 

3,575 

3,131 

-0.39 

0.07 

-0.24 

0.18 

0.16 

0.25 

-0.12 

0.06 

0.53 

18 

Georgia 

3,327 

2,920 

-0.22 

0.02 

0.05 

0.07 

-0.17 

0.11 

0.01 

0.02 

0.52 

8 

Idaho 

5,554 

7,487 

-0.11 

0.06 

-0.19 

0.07 

-0.26 

0.27 

0.01 

0.03 

0.25 

9 

Illinois 

3,761 

3,364 

-0.36 

0.02 

-0.31 

0.08 

-0.17 

0.09 

0.03 

0.02 

0.59 

15 

Indiana 

2,814 

2,344 

-0.47 

0.02 

0.07 

0.06 

-0.11 

0.09 

0.01 

0.02 

0.69 

12 

Iowa 

3,269 

3,206 

-0.31 

0.04 

-0.15 

0.06 

-0.28 

0.11 

0.09 

0.02 

0.37 

12 

Kansas 

4,372 

4,347 

-0.34 

0.03 

-0.06 

0.05 

-0.03 

0.11 

0.14 

0.02 

0.40 

14 

Kentucky 

3,226 

3,797 

-0.33 

0.04 

-0.20 

0.27 

-0.19 

0.15 

0.11 

0.04 

0.34 

21 

Louisiana 

4,175 

5,292 

-0.49 

0.06 

-0.40 

0.64 

-0.48 

0.32 

0.03 

0.06 

0.55 

23 

Maine 

6,296 

4,874 

-0.46 

0.04 

-0.19 

0.13 

-0.57 

0.24 

0.08 

0.04 

0.62 

23 

Maryland 

3,842 

5,444 

-0.53 

0.06 

-0.57 

0.33 

0.11 

0.38 

0.17 

0.05 

0.85 

6 

Massachusetts 

4,253 

4,016 

-0.67 

0.04 

0.61 

0.14 

-0.35 

0.18 

0.02 

0.03 

0.77 

21 

Michigan 

2,771 

2,819 

-0.38 

0.02 

-0.07 

0.06 

-0.56 

0.10 

0.10 

0.02 

0.55 

15 

Minnesota 

3,580 

3,803 

-0.47 

0.04 

0.00 

0.08 

-0.18 

0.12 

0.10 

0.03 

0.44 

20 

Mississippi 

3,687 

3,068 

-0.72 

0.04 

0.67 

0.41 

0.52 

0.23 

0.14 

0.04 

0.73 

27 

Missouri 

3,489 

3,860 

-0.38 

0.02 

-0.23 

0.14 

-0.12 

0.12 

0.09 

0.02 

0.41 

34 

Nebraska 

4,946 

4,932 

-0.09 

0.04 

0.02 

0.08 

0.06 

0.09 

0.02 

0.03 

0.05 

10 

Nevada 

6,731 

4,274 

-0.28 

0.11 

-0.64 

0.46 

-0.09 

0.59 

0.00 

0.06 

0.62 

11 

New 

Hampshire 

6,862 

7,994 

-0.58 

0.06 

0.25 

0.51 

-0.38 

0.17 

0.16 

0.03 

0.67 

13 

New Jersey 

5,099 

2,922 

-0.45 

0.03 

0.11 

0.12 

-0.23 

0.09 

-0.10 

0.03 

0.81 

11 
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TABLE A2 CONTINUED 



Range of adjusted 
spending 





Coefficients and R-squared for 
Predicted Efficiency index 

Achievement 

gains*** 

Between all 
districts in 
the state* 

Between 
all top 
achievers** 

Free and reduced 
lunch 

English-language 

learners 

Special education 

Per-pupil 

expenditure 

R- 

squared 

Percent achievement 
increase if a district 
moves from least to 
most productive, all 
else equal 

Coef- 

ficient 

Standard 

error 

Coef- 

ficient 

Standard 

error 

Coef- 

ficient 

Standard 

error 

Coef- 

ficient 

Standard 

error 

New Mexico 

$6,109 

$7,422 

-0.21 

0.06 

-0.17 

0.06 

-0.06 

0.32 

0.02 

0.04 

0.47 

38% 

New York 

7,326 

7,372 

-0.41 

0.01 

-0.01 

0.06 

-0.36 

0.08 

0.00 

0.01 

0.67 

13 

North Carolina 

3,604 

3,513 

-0.52 

0.04 

0.12 

0.11 

0.27 

0.28 

0.08 

0.04 

0.58 

15 

North Dakota 

4,327 

3,389 

-0.10 

0.13 

-0.74 

0.25 

-0.43 

0.31 

-0.21 

0.09 

0.50 

26 

Ohio 

3,250 

4,081 

-0.31 

0.02 

0.08 

0.07 

-0.27 

0.07 

0.01 

0.02 

0.55 

11 

Oklahoma 

3,889 

4,282 

-0.24 

0.03 

-0.11 

0.06 

-0.18 

0.10 

0.03 

0.02 

0.28 

16 

Oregon 

4,157 

6,740 

-0.29 

0.06 

0.04 

0.07 

-0.33 

0.22 

0.10 

0.03 

0.30 

17 

Pennsylvania 

4,128 

4,309 

-0.42 

0.02 

-0.40 

0.13 

-0.40 

0.10 

0.04 

0.02 

0.61 

15 

Rhode Island 

4,202 

2,986 

-0.57 

0.11 

0.24 

0.45 

-0.69 

0.38 

0.08 

0.08 

0.88 

20 

South Carolina 

2,662 

2,803 

-0.43 

0.04 

-0.01 

0.42 

0.38 

0.20 

-0.09 

0.05 

0.72 

12 

South Dakota 

4,479 

3,772 

-0.27 

0.05 

-0.60 

0.19 

-0.29 

0.10 

0.00 

0.05 

0.65 

15 

Tennessee 

2,652 

2,835 

-0.17 

0.03 

-0.43 

0.19 

0.02 

0.06 

0.02 

0.03 

0.27 

8 

Texas 

6,943 

7,402 

-0.15 

0.01 

-0.29 

0.03 

-0.23 

0.08 

-0.02 

0.01 

0.28 

13 

Utah 

5,499 

5,176 

-0.12 

0.10 

-0.54 

0.17 

-0.38 

0.44 

0.08 

0.05 

0.42 

14 

Virginia 

4,106 

2,424 

-0.16 

0.03 

-0.05 

0.06 

0.30 

0.18 

-0.02 

0.03 

0.25 

9 

Washington 

4,042 

3,632 

-0.47 

0.04 

0.11 

0.07 

-0.17 

0.24 

0.09 

0.03 

0.55 

24 

West Virginia 

2,392 

2,537 

-0.33 

0.06 

-0.64 

0.68 

0.02 

0.22 

0.10 

0.06 

0.38 

10 

Wisconsin 

3,679 

3,340 

-0.28 

0.03 

-0.40 

0.10 

-0.36 

0.14 

0.04 

0.03 

0.50 

11 

Wyoming 

8,606 

4,687 

0.07 

0.12 

-0.67 

0.13 

0.01 

0.33 

-0.10 

0.07 

0.69 

15 


*We defined range of spending in the state as the difference between districts at the 5th and 95th percentiles of the per-pupil-expenditure distribution. Data were adjusted for student needs and cost 
of living. 

**We defined range of spending in the state as the difference between districts at the 5th and 95th percentiles of the per-pupil-expenditure distribution for the districts scoring in the top third of all 
districts in the state. Data were adjusted for student needs and cost of living. 

***We relied on the Predicted Efficiency index to calculate these figures. 
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Appendix B: Frequently 
asked questions 


What is educational productivity? 

In the business world, productivity is a measure of benefit received relative to 
spending. This project adopts that concept to measure public school districts’ 
academic achievement relative to their educational spending, while controlling for 
cost of living, student poverty, the percentage of students in special education, and 
the percentage of English-language learners. 


Why do you say that your evaluations should be approached with 
caution? 

The connection between spending and achievement is complex, and our data can- 
not capture everything that goes into creating an efficient school system. Nor can 
we control for everything that’s outside a district’s control, and our adjustments 
for factors like poverty and students in special education are estimations and don’t 
account for variations in severity and type within those demographic groups. Also, 
some of the data reported by states and districts are unreliable; agencies occasion- 
ally use inconsistent definitions and weak data collection practices. So while we 
believe our results are meaningful, we caution against reading too closely into 
individual evaluations of districts. 


Should the United States spend less on public education? 

Our emphasis on educational productivity does not mean that we believe that 
lawmakers should spend less on education. Quite the opposite. Transforming our 
schools will demand both real resources and real reform, and our project is an 
argument for dramatically improving our nation’s school system so that dollars 
create results. 
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Why didn't you create a single score for each district? 

A single score would have masked wide variation in the rankings of districts 
across our three models. We produced three productivity measures because 
we wanted to emphasize the complexity of measuring a district’s efficiency and 
expose educators, policymakers, and the public to different ways of measuring 
educational productivity. 


Did you evaluate districts against a benchmark? 

No. We evaluated each district relative to the performance of other districts in the 
same state. That means that states with fewer districts have different evaluative 
cut points than states with larger numbers of districts. We believe this approach, 
which has been used in other education policy reports, is a fair way to evaluate 
within-state performance. 


How did you measure achievement? 

We relied on the New America Foundation’s Federal Education Budget Project, 
which collects data from the states on district-level student outcomes. We used 
this data to create an achievement index for each state by assigning each district 
a score. We derived the score by averaging together the percentage of students 
in 2008 designated proficient or above on statewide reading and math tests for 
fourth grade, eighth grade, and high school. 


How did you measure expenditures? 

We used 2008 expenditure data from the National Center for Education Statistics, 
the most recent year for which complete data are available. We used “current 
expenditures,” the preferred metric among educational leaders, which includes 
salaries, services, and supplies. We did not use “total expenditures,” which also 
includes capital expenses, because these can fluctuate dramatically from year to 
year and are thus unreliable for comparisons. 
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How did you account for differences in revenue sources? 

We did not. The fiscal database produced by NCES does not track educational 
expenditures by specific revenue source. 


How did you adjust for differences in cost of living between districts? 

We used the Comparable Wage Index, a measure of regional variations in the 
salaries of college graduates who are not educators. Lori L. Taylor at Texas A&M 
University and William J. Fowler at George Mason University developed the CWI 
to help researchers make better comparisons across geographic areas. We used 
adjustments from 2005, the most recent available. 


Did you adjust for enrollment or economies of scale? 

Apart from excluding from our study any district serving fewer than 250 students, 
we did not adjust for economies of scale because it is difficult to fairly deploy such 
adjustments across state and district lines. There is also debate within the research 
community over what economies of scale say about the quality of a district’s 
management. But given the potential impact that size and location can have on a 
district’s spending, we made it easy to sort by enrollment and geography on our 
interactive website. 


Why did you use the percentage of students at or above the 
"proficient" rather than "basic" level to create your achievement index? 

The proficient level indicates a firm grasp of the knowledge and skills needed to 
succeed at grade level. Students scoring at the basic level have only partially mas- 
tered the necessary knowledge and skills. 


My district scores well on standardized tests, so why does it do 
poorly on your Basic and Adjusted Return on Investment indexes? 

We rate schools on how much academic achievement they get for each dollar 
spent, while controlling for factors outside a district’s control, such as cost of living 
and students in poverty. A district therefore received high marks on our basic and 
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adjusted ROI indexes if it had both high achievement and low spending relative to 
other districts in the same state. Districts with high achievement and high spending 
by definition fare less well, as do districts with low achievement and low spending. 


My district scores poorly on standardized tests. Can it do well on 
your Basic and Adjusted Return on Investment indexes? 

No. School districts with low student achievement cannot get a color rating 
higher than orange — or just below average — on either the basic or the adjusted 
ROI indexes. 


My district scores poorly on standardized tests, so why does it do 
so well on your Predicted Efficiency index evaluation? 

The Predicted Efficiency Index measures whether district achievement is higher 
or lower than its predicted achievement given per-pupil spending and percent- 
age of students in special programs, such as subsidized school lunches. Under 
this approach, a low-achieving district could get high marks if it performed bet- 
ter than expected. 


Can I compare districts across states? 

Because each state has its own student assessment program, the return-on-invest- 
ment measures listed on our website are restricted to within-state comparisons 
of districts, and comparisons across states are not meaningful. We were able, 
however, to conduct a special cross-state analysis of the urban districts that partici- 
pated in the Trial Urban District Assessment (TUDA). The assessment is the only 
source of comparable student performance data at the district level across states, 
and the results of that analysis are listed in the paper. 


Why is my district not included in your evaluation? 

We restricted our study to districts that teach kindergarten through the 12th grade 
and that serve more than 250 students. We also excluded districts classified as 
a charter school agency, state-operated institution, regional education services 
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agency, supervisory union, or federal agency These restrictions were to ensure 
that districts were comparable to one another. We also excluded districts with 
inadequate demographic, achievement, or expenditure data. 


Why is my state not included in your evaluation? 

We did not produce results for Alaska, the District of Columbia, Hawaii, Montana, 
and Vermont. D.C. and Hawaii have only one school district, so within-state com- 
parisons are not possible. Montana and Vermont likewise did not have enough 
comparable districts for meaningful results. We excluded Alaska because we could 
not sufficiently adjust cost-of-living differences within the state. 
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